Optimal projection of observations in a Bayesian setting

Optimal dimensionality reduction methods are proposed for the Bayesian inference of a Gaussian linear model with additive noise in presence of overabundant data. Three different optimal projections of the observations are proposed based on information theory: the projection that minimizes the Kullback-Leibler divergence between the posterior distributions of the original and the projected models, the one that minimizes the expected Kullback-Leibler divergence between the same distributions, and the one that maximizes the mutual information between the parameter of interest and the projected observations. The first two optimization problems are formulated as the determination of an optimal subspace and therefore the solution is computed using Riemannian optimization algorithms on the Grassmann manifold. Regarding the maximization of the mutual information, it is shown that there exists an optimal subspace that minimizes the entropy of the posterior distribution of the reduced model; a basis of the subspace can be computed as the solution to a generalized eigenvalue problem; an a priori error estimate on the mutual information is available for this particular solution; and that the dimensionality of the subspace to exactly conserve the mutual information between the input and the output of the models is less than the number of parameters to be inferred. Numerical applications to linear and nonlinear models are used to assess the efficiency of the proposed approaches, and to highlight their advantages compared to standard approaches based on the principal component analysis of the observations

To be or not to be intrusive? The solution of parametric and stochastic equations - the "plain vanilla" Galerkin case

In parametric equations - stochastic equations are a special case - one may want to approximate the solution such that it is easy to evaluate its dependence of the parameters. Interpolation in the parameters is an obvious possibility, in this context often labeled as a collocation method. In the frequent situation where one has a "solver" for the equation for a given parameter value - this may be a software component or a program - it is evident that this can independently solve for the parameter values to be interpolated. Such uncoupled methods which allow the use of the original solver are classed as "non-intrusive". By extension, all other methods which produce some kind of coupled system are often - in our view prematurely - classed as "intrusive". We show for simple Galerkin formulations of the parametric problem - which generally produce coupled systems - how one may compute the approximation in a non-intusive way

Tensor-based methods for numerical homogenization from high-resolution images

International audienceWe present a complete numerical strategy based on tensor approximation techniques for the solution of numerical homogenization problems with geometrical data coming from high resolution images. We first introduce specific numerical treatments for the translation of image-based homogenization problems into a tensor framework. It includes the tensor approximations in suitable tensor formats of fields of material properties or indicator functions of multiple material phases recovered from segmented images. We then introduce some variants of proper generalized decomposition (PGD) methods for the construction of tensor decompositions in different tensor formats of the solution of boundary value problems. A new definition of PGD is introduced which allows the progressive construction of a Tucker decomposition of the solution. This tensor format is well adapted to the present application and improves convergence properties of tensor decompositions. Finally, we use a dual-based error estimator on quantities of interest which was recently introduced in the context of PGD. We exhibit its specificities when it is used for assessing the error on the homogenized properties of the heterogeneous material. We also provide a complete goal-oriented adaptive strategy for the progressive construction of tensor decompositions (of primal and dual solutions) yielding to predictions of homogenized quantities with a prescribed accuracy

Contributions aux méthodes de calcul basées sur l'approximation de tenseurs et applications en mécanique numérique

This thesis makes several contributions to the resolution of high dimensional problems in scientific computing, particularly for uncertainty quantification. We consider here variational problems formulated on tensor product spaces. We first propose an efficient preconditioning strategy for linear systems solved by iterative solvers using low-rank tensor approximations. The preconditioner is found as a low rank approximation of the inverse of the operator. A greedy algorithm allows to compute this approximation, possibly imposing symmetry or sparsity properties. This preconditioner is validated on symmetric and non-symmetric linear problems. We also make contributions to direct tensor approximation methods which consist in computing the best approximation of the solution of an equation in a set of low-rank tensors. These techniques, sometimes coined "Proper Generalized Decomposition" (PGD), define optimality with respect to suitable norms allowing an a priori approximation of the solution. In particular, we extend the classically used greedy algorithms to the sets of Tucker tensors and hierarchical Tucker tensors. To do so, we construct successive rank one corrections and update the approximation in the previous sets. The proposed algorithm can be interpreted as a construction of an increasing sequence of reduced spaces in which we compute a possibly approximate projection of the solution. The application of these methods to symmetric and non-symmetric problems shows the efficiency of this algorithm. The proposed preconditioner is also applied and allows to define a better norm for the approximation of the solution. We finally apply these methods to the numerical homogenization of heterogeneous materials whose geometry is extracted from images. We first present particular treatment of the geometry and the boundary conditions to use them in the tensor approximation framework. An adaptive approximation procedure based on an a posteriori error estimator is proposed to ensure a given accuracy on the quantities of interest which are the effective properties. The method is first developed to estimate effective thermal properties, and is extended to linear elasticity.Cette thèse apporte différentes contributions à la résolution de problèmes de grande dimension dans le domaine du calcul scientifique, en particulier pour la quantification d'incertitudes. On considère ici des problèmes variationnels formulés dans des espaces produit tensoriel. On propose tout d'abord une stratégie de préconditionnement efficace pour la résolution de systèmes linéaires par des méthodes itératives utilisant des approximations de tenseurs de faible rang. Le préconditionneur est recherché comme une approximation de faible rang de l'inverse. Un algorithme glouton permet le calcul de cette approximation en imposant éventuellement des propriétés de symétrie ou un caractère creux. Ce préconditionneur est validé sur des problèmes linéaires symétriques ou non symétriques. Des contributions sont également apportées dans le cadre des méthodes d'approximation directes de tenseurs qui consistent à rechercher la meilleure approximation de la solution d'une équation dans un ensemble de tenseurs de faibles rangs. Ces méthodes, parfois appelées "Proper Generalized Decomposition" (PGD), définissent l'optimalité au sens de normes adaptées permettant le calcul a priori de cette approximation. On propose en particulier une extension des algorithmes gloutons classiquement utilisés pour la construction d'approximations dans les ensembles de tenseurs de Tucker ou hiérarchiques de Tucker. Ceci passe par la construction de corrections successives de rang un et de stratégies de mise à jour dans ces ensembles de tenseurs. L'algorithme proposé peut être interprété comme une méthode de construction d'une suite croissante d'espaces réduits dans lesquels on recherche une projection, éventuellement approchée, de la solution. L'application à des problèmes symétriques et non symétriques montre l'efficacité de cet algorithme. Le préconditionneur proposé est appliqué également dans ce contexte et permet de définir une meilleure norme pour l'approximation de la solution. On propose finalement une application de ces méthodes dans le cadre de l'homogénéisation numérique de matériaux hétérogènes dont la géométrie est extraite d'images. On présente tout d'abord des traitements particuliers de la géométrie ainsi que des conditions aux limites pour mettre le problème sous une forme adaptée à l'utilisation des méthodes d'approximation de tenseurs. Une démarche d'approximation adaptative basée sur un estimateur d'erreur a posteriori est utilisée afin de garantir une précision donnée sur les quantités d'intérêt que sont les propriétés effectives. La méthodologie est en premier lieu développée pour l'estimation de propriétés thermiques du matériau, puis est étendue à l'élasticité linéaire

Weakly intrusive low-rank approximation method for nonlinear parameter-dependent equations

International audienceThis paper presents a weakly intrusive strategy for computing a low-rank approximation of the solution of a system of nonlinear parameter-dependent equations. The proposed strategy relies on a Newton-like iterative solver which only requires evaluations of the residual of the parameter-dependent equation and of a preconditioner (such as the differential of the residual) for instances of the parameters independently. The algorithm provides an approximation of the set of solutions associated with a possibly large number of instances of the parameters, with a computational complexity which can be orders of magnitude lower than when using the same Newton-like solver for all instances of the parameters. The reduction of complexity requires efficient strategies for obtaining low-rank approximations of the residual, of the preconditioner, and of the increment at each iteration of the algorithm. For the approximation of the residual and the preconditioner, weakly intrusive variants of the empirical interpolation method are introduced, which require evaluations of entries of the residual and the preconditioner. Then, an approximation of the increment is obtained by using a greedy algorithm for low-rank approximation, and a low-rank approximation of the iterate is finally obtained by using a truncated singular value decomposition. When the preconditioner is the differential of the residual, the proposed algorithm is interpreted as an inexact Newton solver for which a detailed convergence analysis is provided. Numerical examples illustrate the efficiency of the method

Approximation de faible rang d'un échantillon de solution d'une équation paramétrée

International audienceUn échantillon de solution d'une équation paramétrée est approché par un tenseur de faible rang. On s'intéresse tout d'abord à l'approximation en norme L2 et L∞ d'un échantillon d'apprentissage. Ensuite, plusieurs méthodes sont envisagées en fonction des informations disponibles sur la structure de l'équation. Elles s'appuient toutes sur l'accès au résidu, mais sont raffinées si des informations complémentaires sont accessibles

Approximation de faible rang de l'inverse d'un opérateur pour le pré-conditionnement de systèmes linéaires sur un espace produit tensoriel

International audienceDes préconditionneurs sont proposés pour les systèmes linéaires sur un espace produit tensoriel. Ils sont définis comme des approximations de faible rang de l'inverse de l'opérateur et obtenus par la formulation d'un problème de meilleure approximation au sens d'une norme adaptée. Les pré-conditionneurs sont appliqués aux solveurs itératifs couplés aux méthodes d'approximation de tenseurs, ainsi qu'aux méthodes d'approximation directe de la solution dans des ensembles de tenseurs de faible rang. Cette stratégie est illustrée sur des exemples numériques tels que la résolution d'une équation de diffusion en grande dimension ou d'une équation paramétrique pour la quantification d'incertitudes

To Be or Not to be Intrusive? The Solution of Parametric and Stochastic Equations---Proper Generalized Decomposition

International audienc

Predictive distributions of random variables following a multivariate Gaussian distribution with Normal Inverse Wishart prior: Technical report

The Gaussian distribution is a popular choice of density for the modeling of continuous observations, often combined with a specific prior in the context of Bayesian inference. When this prior is conjugate (e.g., the normal-gamma inverse distribution in the univariate case), it can be proved that the posterior and predictive distributions have analytically tractable expressions. While these derivations are readily available in the literature for the univariate case, the multivariate case derivations are significantly harder to find. This short technical note proposes a detailed derivation of these closed-forms in the multivariate case with a conjugate Normal Inverse Wishart prior. Keywords Bayesian Inference • Predictive Distributions • Multivariate Gaussian • Normal Inverse Wishart The Gaussian distribution is a popular choice of density for the modeling of continuous observations, often combined with a specific prior in the context of Bayesian inference. When this prior is conjugate (e.g., the normal-gamma inverse distribution in the univariate case), it can be proved that the posterior and predictive distributions have analytically tractable expressions. The most cited source [1] (e.g., cited in [2, 3]) proves the equivalence in the univariate case. However, the multivariate case derivations are significantly harder to find. In the following we propose a detailed derivation of the multivariate distribution, based on the exponential family expressions of the Gaussian and Normal Inverse Wishart distributions